Principles for an Alternative Design of Movement Primitives that Uses Probabilistic Inference in Learned Graphical Models

For motor skill learning in high-dimensional continuous action spaces we propose to endow movement representations with an intrinsic probabilistic planning system, integrating the power of stochastic optimal control methods within a movement primitive. The parametrization of the primitive is a graphical model that represents the dynamics and constraints, encoded by an intrinsic cost function, such that inference in this graphical model yields the control policy. We parametrize the intrinsic cost function using task-relevant features, such as the importance of passing through certain viapoints. The system dynamics as well as intrinsic cost function parameters are learned in a reinforcement learning setting. This alternative movement representation complies with salient features of biological movement generation, i.e. its modular organization in elementary movements, its characteristics of stochastic optimality under perturbations, and its efficiency in terms of learning [1].